Top data engineering writer on Medium & Senior Data Engineer in media; I use my skills as a former journalist to demystify data science/programming concepts so beginners to professionals can target, land and excel in data-driven roles.
You Can Build A Data Pipeline In <90 Min.
Extract. Transform. Read.A newsletter from Pipeline: Your Data Engineering ResourceHi past, present and future data professional! Since today is a U.S. holiday, I won’t take much of your time; the good news is that, when conducted efficiently, building a data pipeline doesn’t have to take days, weeks or months. In fact, you can build a data pipeline in as little as 90 minutes. Accelerating pipeline development depends on a thorough read of the documentation, a familiarity with your scripting language’s requests library and patience dealing with pesky data structures. If you think, during this time, engineers are heads-down, you may have watched The Social Network too many times; personally, I like a little external stimuli while coding, which is how I ended up building a full dashboard during another American pastime–a baseball game. My secret? Distilling data with clean views, which I recommend over bloated source tables for both aesthetic and performance reasons. Even optimizations like views have their limitations, leading to optimization ceilings. The best way to break through, aside from stubbornness, is a combination of incremental problem-solving and “big picture” data modeling to reassess resources and attack the problem completely. Since I don’t want you to have to work any harder today, here are the embedded links as text:
If you’re celebrating America today, happy 4th! Thanks for ingesting, -Zach |
Pipeline To DE
Top data engineering writer on Medium & Senior Data Engineer in media; I use my skills as a former journalist to demystify data science/programming concepts so beginners to professionals can target, land and excel in data-driven roles.
Extract. Transform. Read. A newsletter from Pipeline Hi past, present or future data professional! When you apply to data analysis, data engineering or data science jobs, you likely consider factors like company name, culture and compensation. Caught up in the excitement of a fresh opportunity or compelling offer you’re neglecting an important part of your day-to-day reality in a new role: What stage of data maturity the organization is in. If you’re looking for experience building something...
Extract. Transform. Read. A newsletter from Pipeline Sponsored by Basejump AI. Don't just query your data; chat with it. Learn more. Hi past, present or future data professional! If you’re a hybrid or remote worker (or online student) you know the FOMO that sets in when you leave your desk and something at work needs your immediate attention. Even though data engineering is a role that can be done from anywhere, having to open, set up and position a laptop isn’t ideal in all situations like...
Extract. Transform. Read. A newsletter from Pipeline: Your Data Engineering Resource Presented by Basejump AI Use natural language prompts to chat with your database in Basejump’s intuitive interface, or embed it directly in your application. Book your demo here. Hi past, present or future data professional! One thing that makes my work day easier is when I’m Google-ing (as all software developers do) a problem and I come across the holy grail of solutions: A one-line implementation. Like...